Structure-preserving desynchronization of minority games 
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Perfect synchronicity in iV-player games is a useful theoretical dream, but communication delays 
are inevitable and may result in asynchronous interactions. Some systems such as financial markets 
are asynchronous by design, and yet most theoretical models assume perfectly synchronized actions. 
We propose a general method to transform standard models of adaptive agents into asynchronous 
systems while preserving their global structure under some conditions. Using the Minority Game as 
an example, we find that the phase and fluctuations structure of the standard game subsists even 
in maximally asynchronous deterministic case, but that it disappears if too much stochasticity is 
added to the temporal structure of interaction. Allowing for heterogeneous communication speeds 
and activity patterns gives rise to a new information ecology that we study in details. 

PACS numbers: 89.65.Gh, 89.75.Fb, 64.60.De 



I. INTRODUCTION 

When a large number of agents taking part in a multi- 
player game submit their actions to a central authority 
(game server, financial market), the times at which their 
actions become effective are likely to differ because of 
reaction times, transmission delays or backlogs at the 
central server. If the resulting delay is sufficiently large, 
the synchronicity of actions and payoffs is not a reason- 
able assumption anymore. Yet the immense majority of 
the literature on games and agents assumes perfect syn- 
chronicity (see however e.g. [l|, 0, Q for some notable 
exceptions) . 

This is problematic in the modelling of many systems, 
among which the numerous financial markets where the 
actions of agents arc discrete in time and asynchronous. 
Most financial market models aggregate traders' actions 
over a given period in one time step. Unless it corre- 
sponds to sensible time periods, such as one trading day, 
this approach is rather artificial. While time coarsening 
simplifies the description of market dynamics, specula- 
tion cannot be modelled by including all the agents' ac- 
tions in one time step: one does not make money with 
a single transaction, i.e. in a single time step. In ad- 
dition, the emergence of large price and volume fluctua- 
tions must also be explained in an asynchronous setting. 
Finally asynchronicity also originates from the hetero- 
geneity of time scales of market participants, which is 
fat-tailed, possibly a power-law 0, H, @ . 

This raises fundamental issues regarding macroscopic 
synchronization, especially in the case of coordination 
and cooperation. Here we modify the well-understood 
minority game Q by introducing a tunable time delay 
between the submission of a bid and its actual influence 
on the global outcome, and a tunable playing frequency. 

Remarkably the structure of mean-field models such 
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FIG. 1: Schematic explanation of playing period T, time delay 
S, bids b(t) and actions a(t + S). 



as the MG is preserved; as a consequence there is hope 
that the resulting asynchronous interaction somehow still 
belongs to the mean-field category, hence, that the pow- 
erful methods from statistical mechanics that solve the 
original model [8|, |9( can be generalized. 



II. DESYNCHRONIZING GLOBAL GAMES 

Let us consider a global synchronous game. Each agent 
i = 1, • • • , N takes action ai{t) at time t. His payoff is a 
function of his own action and of the global action of all 
the agents A(t) = X^j=i a j(t) : everything can be written 
as a function of time t only. 

A way to desynchronize the agents while preserving 
the global structure of the game is to think in terms of 
delays: if agent i submits his action at time t, the latter 
becomes effective (i.e. is incorporated into A) only at 
time t + S. But at time t + 6, A(t + 5) only contains the 
actions effective at that time, that is, does not contain 
the action of some other agents that are still being trans- 
mitted or thought about. Thus, the payoffs of the agents 
at this time do not reflect the most recent actions, but 
only the last known actions. Differentiating between ac- 
tions sent but not effective yet, thereafter called bids and 
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denoted by and actual actions cii{t) makes it easy 

to introduce asynchronous actions while keeping intact 
the payoff structure, as will be shown in the next section 
(see Fig. [I]). 

Mathematically, agent i submits his bid bi(t); between 
times t and t + S—1, his last known action is unchanged; 
at time t+S, the last bid becomes effective, i.e. a,i(t+5) = 
bi(t) and of course A(t + 6) = £\ a i(t + <^)i agent % then 
receives his payoff that depends on di(t + S) and A(t + S). 
In this way, a non-trivial structure of overlapping bids 
and actions can be built. 

The temporal structure of the game is completely spec- 
ified by assuming that agent % is active at times t\ , 

(n) 

n > 0. For instance, t\ nTi + (pi where n is an inte- 
ger. The maximally asynchronous case corresponds to 
Ti = N and <pi — i: only one agent is active at each time 
step. Alternatively, an agent may be active with some 
probability at each time step, thereby removing the rigid 
structure imposed by periodic t 
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III. EXAMPLE: MINORITY GAMES 

The minority game (MG thereafter) is a prototype 
model of global competition between adaptive agents 
0, HI • Well- understood [1, |j| , it provides an ideal test- 
bed for new ideas and extensions. At the same time, 
it has highly non-trivial and characteristic fluctuations 
structure and phase transition. The existence of a phase 
transition with symmetry breaking (ToL [ill ] is robust with 
respect to a surprising number and types of modifications 
BE!; as sucn j if present in modified games, it is a sig- 
nature that the original dynamics has not been overly 
altered. 

The aim of the agents is, as the name of the game 
implies, to be in the minority: in the original game, agent 
i takes action dj(t) € { — 1,1}; A(t) is defined as above 
and is positive if the majority chose +1 and vice- versa. 
The payoff of agent i is — a,i(t)A(t): those who happen to 
be in the minority are rewarded. 

The various types of minority games found in the lit- 
erature differ mostly in their learning and decision mech- 
anisms. The original one is defined as follows: agents 
are fed with the last m winning decisions, a bitstring 
called history and denoted by n(t). Each agent has a 
set of S strategies, i.e., of predefined ways to react to 
all possible public pieces of information. Denoting the 
strategies of agent i by di )S , s = 1, • • • , S, one can rewrite 

A(t) = J2i a i^s\t) wnere s i(t) is the strategy trusted by 
agent i at time t. 

Which strategy to choose is determined by reinforce- 
ment learning. To this effect, since the agents gather 
information about the world through the use of their 
strategies, they store experience about the past in vir- 
tual performance scores of their strategies that evolve 



U itS {t + l) = U us {t) 



A(t) 



(1) 



The agents choose their best strategy at time t. In other 
words, Sj(t) = argmax s Ui iS (t). 

When desynchronizing the game, a slight complication 
with respect to game histories arises: whereas in perfectly 
synchronized games n(t) is the same for all the agents, 
this cannot hold anymore in asynchronous settings since 
all the agents do not see the same A when they receive a 
payoff. Thus each agent has his own history of the game, 
which encodes the past m right choices for him. 

By construction (A) = 0. We shall be interested in the 
fluctuations of the global outcome a 1 — (A 2 ) which quan- 
tify the degree of coordination of the players, the bench- 
mark being the random outcome a 2 — N . Predictability 
must be measured at the individual level: one defines the 
conditional average of the attendance from the point of 
view of agent i, that is, conditional to his histories, which 
we denote as (j4.W|/Xj), and the predictability as seen by 
this agent Hi = ^ (A® \fii) 2 /P, where P = 2 M , while 
the average predictability is H = J^i Hi/N. 

Predictability H > corresponds to a (conditional) 
symmetry breaking between the two choices. The stan- 
dard game is characterized by a predictable phase (H > 
0) for a — P/N > a c , an unpredictable phase (H = 0, 
a < a c ) and critical point a c = 0.3374 . . . where H — > 
and <r 2 /N reaches a minimum d, H, [l^, [H|. The pres- 
ence of this phase transition is acknowledged to be robust 
with respect to many modifications of the game, except 
when all the agents take into account their impact on 
the game by removing their contribution to A, i.e. , by 
replacing A with A^ = A - a t (t) in Eq Q 0, tT||. 
However, little is known about the importance of the 
synchronization of histories with respect to the existence 
of the phase transition. Local games are synchronized 
MGs where an agent plays against his neighbors, giving 
rise to partially spatially overlapping games, hence, his- 
tories [l4l [l5| . whereas the overlaps are in time in our 
case. 

Another issue is the information ecology: given the fact 
that predictability is easily measured, MGs give insights 
on who exploits whom [lq . that is, in this case, on the 
risk associated with delays, for instance. Since the MG 
is a negative sum game, making an average positive gain 
is hard; it is only achieved in the original game by some 
agents that can exploit very efficiently the predictabil- 
ity left by other agents, such as e.g. some of those able 
to settle on one strategy in the standard game (frozen 
agents) [l(| El, EH) speculators feeding on producers, 
insider trading or a longer history length deep in the un- 
predictable phase [l6[. Recently two of us designed and 
studied synchronous MGs where the agents had hetero- 
geneous time scales 17J. Here we shall characterize the 



importance of communication delays and frequency of 
play. 
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FIG. 2: Fluctuations a 2 /N (red squares; right scale) and 
predictability H/N (black circles; left scale) of the maxi- 
mally asynchronous MG as a function of a = P/N. M = 4 
(P = 16), 5 = 2, 4007VP iterations, 200iVP time steps until 
equilibration, averages over 200 samples. 
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FIG. 3: Autocorrelation of A(t) in a population with hetero- 
geneous time delays T t ; M = 4 (P = 16), N = 51, S = 2, 
400iVP iterations, 200iVP time steps until equilibration. 



A. Results 

Numerical simulations are about N times slower than 
those of the usual MGs, as one effective time-step from 
a measurement point of view ends when all the agents 
have updated their actions. Even worse, the interesting 
regions are found for quite low a = P/N , which makes 
computations even slower. This unfortunately limits the 
system sizes one can study with current computers to 
M = 4, i.e. P = 16. Indeed one run for the maximally 
asynchronous case at P = 32 and P/N = 0.01 needs 150 
minutes a modern computer (Core 2 duo, 2GHz) , hence 
averaging over 200 samples requires 20 days just for this 
point. Fortunately, M = 4 yields good enough results, 
as discussed in the concluding section. 

1. Maximally asynchronous game 

Assuming that t\ n ' = nTi + fa as above and setting 
Ti = T = N and fa = i, there is only one active agent at 
each time step, hence A changes at most by 2 in a time 
step. In this case, the histories /Lt« of agents 1 and N/2 
will likely differ, unless the dynamics of A has a memory 
longer that N/2. 

Plotting as usual a 2 /N and H/N reveals that this sys- 
tem undergoes the same phase transition as the original 
minority game (see Fig. [2]) . This means that the desyn- 
chronization we propose has the remarkable property to 
keep the global structure of the game unchanged, while 
allowing for extreme asynchronicity. It is notable that the 
predictability is orders of magnitude smaller than a 2 /N, 
which is to be expected since the very long time delays Tj 
of the agents dilutes information; the order of magnitude 
of the fluctuations on the other hand is unchanged, since 
A(x VN. 

The auto-correlation of A reveals a complex pattern, 



similar to that seen in Fig. [3] since A is only possibly 
changed by only one agent at a time, it displays persis- 
tence for 0(N/2) time steps. After this decay, it shows 
on average negative auto-correlation, as can be expected 
in a minority game where every deviation from A = 
tends to be cancelled by adaptive agents. The additional 
oscillations are of period N. 



2. Heterogeneous time delays 

The pattern of auto-correlation of A suggest that play- 
ing with a high frequency allows one to take profit from 
the persistence of A. 

Drawing at random Ti from {1, • • • , pN} and setting 
Si = T i: allows one to study the respective gains associ- 
ated with a time scale denoted by (g\T) = (g\S), in order 
words, the information ecology that arises from being 
active more often and having a shorted delay in an asyn- 
chronous setting. Figure [1] shows that in asynchronous 
situations being faster is an clear advantage, a few of the 
agents reaping even positive gains. 

Positive gains come quite peculiarly for minority games 
from the unconditional persistence of A. Computing the 
autocorrelation. (A(t)A(t + r)) (Fig. [3]) shows a memory 
of A lasting N/2 time steps, which corresponds exactly to 
the point where the losses of the agents saturates in Fig 
[J] Earning a positive gain, though, requires enough per- 
sistence to overcome the cost of play the MG, a negative 
sum game. 



3. Activity frequency and time delays 

In the two previous subsections, one assumed that the 
frequency of activity of a given agent is exactly equal to 
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FIG. 4: Average gain of agents with a given time delay Ti 
versus T; M = 4 (P = 16), TV = 101, 5 = 2, 400TVP itera- 
tions, 200TVP time steps until equilibration, averages over 10 5 
samples. 
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FIG. 6: Predictability H/N and fluctuations a 2 /N as a func- 
tion of q = P/TV for Poissonian activity (9 = 1). Af = 4 
(P = 16), S = 2, 400TVP iterations, 200TVP time steps until 
equilibration, averages over 200 samples. 
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FIG. 5: Contour plot M = 4 (P = 16), N = 101, S = 
2, 400TVP iterations, 200TVP time steps until equilibration, 
averages over 10 5 samples. 



his time delay. This restriction is unrealistic and can be 
lifted easily: assume that an agent plays every Tj time 
steps and has a time delay of Si . The delay Si may happen 
to be larger than Tf. if an agent is unfortunate enough to 
have a larger delay than activity frequency, he waits for 
Si time steps to receive his payoff and sends a new bid at 
the next possible activity time t'f'' . 

The average gains conditional on 8 and T, denoted 
by (g\S,T), reveal in more details the information ecol- 
ogy of asynchronicity: Figure \E\ reports a contour plot of 
(g\S, T); T and S play a similar role: the more frequently 
one is active, the more one can profit from the persistence 
of A, i.e., from the slowness of other players, as before. 
But <5, which can be seen as the quickness of reaction to 
new information, is expectedly the most relevant param- 
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FIG. 7: Fraction of frozen agents (f> and fluctuations a 2 /TV as 
a function of 8/N for Poissonian activity for a single realisa- 
tion of the disorder. M = 4 (P = 16), S = 2, 80000TVP/6> 
iterations, 4000TVP/# time steps until equilibration. 



eter: it mainly controls whether one obtains a positive or 
negative gain, except around i5 = 4, where T plays this 
role; the gain decreases as T increases, which is consis- 
tent with the results of the previous subsection. The last 
feature of the plot is the slight bump at 8 = T, which is 
due to the fact that the players who have S = T + 1 have 
effectively T" — 2T, hence the slight decrease of average 
gain. 
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4- Poissonian activity 

Finally, let us relax completely the periodicity of agent 
activity: each agent i plays now at time t with probability 
9/N; the standard MG is recovered when 9 — N. We first 
take = 1, which is stochastically equivalent to the setup 
of section IIII A 1[ except that the temporal interaction 
has no more structure. Figure [6] suggests that the phase 
transition disappears completely, at least for the range of 
parameters we swept over (we tried also tried individual 
runs at a = 0.001 that confirm this result). 

This means that at least some structure of overlap be- 
tween the histories of the agents is a crucial ingredient of 
the phase transition. If the phase transition is resilient 
to some stochastic perturbation of the personal history 
update temporal structure, one should find a critical 9 C 
for which a system placed in the symmetric phase under- 
goes a phase transition when sweeping over 9. The best 
way to test for a phase transition is to plot the fraction of 
frozen agents, <f>, defined as the fraction of agents which 
played the same strategy during the last half of the time 
steps [11]]. The only situation for which <fi = can occur 
is when H = 0. Figure [7] shows that the system stays in 
the symmetric phase as long as > 9 C /N ~ 0.5 and then 
<j> increases rapidly. The fluctuations drop discontinu- 
ously at 9 = 1: anergordicity is broken by stochasticity, 
but the system stays in a symmetric phase. 

IV. DISCUSSION 

We have proposed a powerful and generic way to desyn- 
chronize global games while preserving their structure. 
The resulting asynchronicity allows for the study of the 
effect of time delays and playing frequency in principle 
in any A^-player game. 

Desynchronized MGs provide yet another example 
of the robustness of the phase transition in MGs (see 
the many other phase transition-preserving modifications 
listed in 0]). We did not study in detail the location of 



this phase transition; additional lengthy numerical sim- 
ulations are needed to study this point in detail, in par- 
ticular as a function of T . 

The fact that the phase transition disappears in the 
presence of strong enough stochastic desynchronization 
is a clue that this robustness relies on the temporal struc- 
ture of individual history updates, as also confirmed by 
the fact that history- less games (P = 1) [l8|, [lj| do not 
reproduce the delay information ecology found in the 
present study. Interestingly, recent work on the Pris- 
oner's Dilemma showed that similar Poissonian desyn- 
chronization leads to a first order phase transition at fi- 
nite activity frequency 20]. 

Although we had to consider quite small systems (M = 
4), the results presented here will not change qualitative 
when simulating much larger systems is doable. Finite 
size effects of minority games are well-studied [|[; two 
points are crucial: i) all the macroscopic variables de- 
pend only on a = 2 M /N, up to finite size effects; ii) the 
existence of the phase transition is found for all M > 1. It 
should be noted that macroscopic variables (H/N, a 2 /N) 
of the MG with real histories may have peculiar finite size 
effects for very small M because of the De Bruijn graph 
on which bitstrings of length M live; when M > 4, its 
complexity is large enough to prevent the histories of the 
game to be stuck in a small trivial subgraph. 

The familiar phase structure of asynchronous games 
suggests to solve this kind of games with mathematical 
methods that have been successfuly applied to the orig- 
inal MGs and some of its extensions @, Q. Two new 
problems arise: first actions are delayed, which makes 
the whole calculus more complex; the second problem 
comes from the fact that each agent has his own real his- 
tory: solving the standard MG with re al (g lobal) histo- 
ries was a mathematical tour de force 0, |21| : solving sys- 
tems with individual histories will need an even more im- 
pressive feat. Nevertheless, there is nothing in principle 
that makes the computation infeasible, although it will 
be much more complex. We hope that such future work 
will reveal the inner dynamical effects of asynchronicity. 
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